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(54) INPUT POSITION MEASURING INSTRUMENT AND ENTERTAINMENT SYSTEM 

(57) An input position detection device for detecting 
an input position of an input device, and an entertain- 
ment system which can detect the position of a subunit 
and has a display which displays an image generated 
by execution of a program loaded into the entertainment 
system. 



The entertainment system has a portable game ter- 
minal 3 held by the operator 7. The portable game ter- 
minal 3 is equipped with an infrared input/output unit 8 
which sends infrared signals corresponding to the com- 
mands of the operator 7 to an infrared input/output 
device 4 which is connected to a monitor device 6. The 
infrared input/output device and the CCD camera 5 are 
disposed adjacent each other and point in the direction 
of the operator 7 holding the portable game terminal 3. 
A game machine main unit 2 detects the position of the 
portable game terminal 3 based on the difference 
between an input image from the CCD camera 5 taken 
immediately before the operator 7, sends an infrared 
signal from the infrared input/output unit 8 of the porta- 
ble game terminal 3 toward the infrared input/output 
device 4, and the input image at the time that the infra- 
red signal is sent. 
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Description 
Technical Field 

[0001] The present invention relates to an input s 
position detection device for detecting an input position 
by means of an input device equipped with first infrared 
transceiving means and an entertainment system which 
includes a main unit that has program execution func- 
tions and a subunit equipped with an interface and infra- io 
red transceiving means for connecting to the main unit. 

Background of the Invention 

[0002] In a video game player machine which is a is 
specific example of an entertainment system, player 
characters and objects and the like are displayed on a 
monitor device such as a television receiver connected 
to the main unit of the game machine, and an input con- 
troller is used to cause the characters or objects to jump 20 
and move or the like as a role-playing game, racing 
game or the like proceeds. 

[0003] The game proceeds assuming that the 
player normally views the monitor device from the front. 
For this reason, the player is required to be positioned in 25 
front of the monitor device. 

[0004] However, depending on the situation of the 
room in which the monitor device is placed, it may be 
difficult for the player to be positioned directly in front of 
the monitor device. 30 
[0005] In addition, when two people each use their 
own input controllers to play a head-to-head type game, 
it may be difficult for both players to be positioned 
directly in front of the monitor device. 
[0006] It is an object of the present invention to pro- 35 
vide an input position detection device that can detect 
the input position of an input device accurately and sim- 
ply, and thus detect the position of the subunit and dis- 
play an image generated by the execution of a program 
along the line of sight centered about that position, 4c 
thereby setting the point of view optimally suited to the 
player. 

Disclosure of the Invention 

45 

[0007] These and other objects are attained by an 
input position detection device for detecting an input 
position by means of an input device equipped with first 
infrared transceiving means, comprising second infra- 
red transceiving means that sends infrared signals to so 
and receives infrared signals from the first infrared 
transceiving means of said input device, and image 
pickup means for picking up an image in a stipulated 
direction. The image pickup means and the second 
infrared transceiving means are disposed adjacently 55 
and point in a stipulated direction, and the input position 
of the input device in the stipulated direction or the vicin- 
ity thereof is detected based on the difference between 



an image picked up by the image pickup means that 
contains an infrared signal for position detection from 
the second infrared transceiving means of the input 
device in the stipulated direction or the vicinity thereof 
and an immediately previous image that does not con- 
tain an infrared signal for position detection. 
[0008] Moreover, in the entertainment system of the 
invention comprising a main unit that has program exe- 
cution functions and a subunit equipped with an inter- 
face and first infrared transceiving means for connecting 
to the main unit, the main unit has a second infrared 
transceiving means that sends and receives infrared 
signals to and from the first infrared transceiving means 
of the subunit, and image pickup means for picking up 
an image in a stipulated direction disposed adjacently 
with the second transceiving means and pointed in a 
stipulated direction, wherein the input position of the 
subunit in said stipulated direction or the vicinity thereof 
is detected based on the difference between an image 
picked up by the image pickup means that contains an 
infrared signal for position detection from the infrared 
transceiving means of the subunit in the stipulated 
direction or the vicinity thereof and an immediately pre- 
vious image that does not contain an infrared signal for 
position detection. 

[0009] Here, the main unit is connected to display 
means that displays images generated by the execution 
of programs, and said infrared transceiving means and 
said image pickup means are disposed near this display 
means. 

[001 0] In addition, the subunit has a unique identifi- 
cation number and the identification number information 
is contained in infrared light emitted toward said main 
unit, in addition, the subunit has a unique identification 
number and when the identification number is sent from 
the main unit to the subunit by means of infrared sig- 
nals, a determination is made as to whether or not the 
identification number is equal to a unique identification 
number. 

Brief Description of the Drawings 
[0011] 

Fig. 1 is a block diagram showing the configuration 
of a video game player machine as an embodiment 
of the input position detection device and entertain- 
ment system according to the present invention; 
Fig. 2 is a block diagram showing the detailed con- 
figuration of parts of the aforementioned video 
game player machine; 

Fig. 3 is a flowchart for explaining the operation of 
the aforementioned video game player machine; 
Fig, 4 is a diagram used for explaining the computa- 
tional algorithm of the image address computer 
constituting the video game player machine; 
Fig. 5 is a block diagram showing the configuration 
of variation of the aforementioned video game 
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player machine; 

Fig. 6 is a flowchart used for explaining the opera- 
tion of the embodiment shown in Fig. 5; and 
Fig. 7 is a diagram for explaining the operation of a 
video game player equipped with two portable 
game terminals. 

Best Mode for Carrying Out the Invention 

[0012] Here follows an explanation of the embodi- 
ment of the input position detection device and enter- 
tainment system according to the present invention with 
reference to the figures of the drawings. This embodi- 
ment is a video game player machine 1 consisting of a 
game machine main unit 2 that has program execution 
functions and a portable game terminal 3 equipped with 
an interface for making a wireless connection to the 
game machine main unit, while the game machine main 
unit is equipped with functions as an input position 
detection device that detects the input position from the 
portable game terminal. 

[001 3] Fig. 1 shows the video game player machine 
1 which consists of the game machine main unit 2 and 
the portable game terminal 3. The game machine main 
unit 2 is connected to an infrared input/output device 4, 
a CCD camera 5 serving as the image pickup means 
and a monitor device 6 such as a television receiver. 
[0014] The portable game terminal 3 is held by an 
operator 7. The portable game terminal 3 is equipped 
with an infrared input/output unit 8 which sends infrared 
signals corresponding to the commands of the operator 
7 to the infrared input/output device 4. 
[0015] Here, the infrared input/output device 4 con- 
nected to the monitor device 6 and the CCD camera 5 
are disposed adjacent to each other and point in the 
direction of the operator 7 holding the portable game 
terminal 3. The game machine main unit 2 detects the 
position of the portable game terminal 3 based on the 
difference between an input image from the CCD cam- 
era 5 taken immediately before the operator 7 sends an 
infrared signal from the infrared input/output unit 8 of the 
portable game terminal 3 toward the infrared input/out- 
put device 4, and the input image at the time that the 
infrared signal is sent. The portable game terminal 3 
has a unique identification number and this identifica- 
tion number information is contained in the infrared sig- 
nal sent toward the game machine main unit 2. 
[0016] As shown in Fig. 2, the game machine main 
unit 2 consists of an A/D converter 10 that converts the 
image picked up by the CCD camera 5 into a digital sig- 
nal, a switch 1 1 that changes positions when an infrared 
signal for position detection is supplied from the infrared 
input/output device 4, two frame buffers 12 and 13; a 
read-out circuit 14, a comparator 15 and an image 
address computer 16. 

[001 7] Next, the operation of the video game player 
machine 1 will be described using the flowchart shown 
in Fig. 3. First, in Step S1, the operator 7 presses the 



infrared signal send button of the portable game termi- 
nal 3. Then, in Step S3, the portable game terminal 3 
sends from the infrared input/output unit 8 an infrared 
signal containing its own identification number. 

s [001 8] The infrared input/output device 4 connected 
to the game machine main unit 2 detects the infrared 
signal from the infrared input/output unit 8 sent in Step 
S3. Then, the signal from the portable game terminal 3 
is determined from the identification number contained 

w in the infrared signal detected in Step S4, and after the 
image picked up by the CCD camera 5 is converted into 
a digital image signal by the A/D converter 10, one 
frame is stored in frame buffer 12. 
[0019] In Step S5, when the infrared signal for posi- 

15 tion detection from the portable game terminal 3 is sent 
for a length of time equal to or longer than one frame, in 
Step S6 the game machine main unit 2 uses the CCD 
camera 5 to pick up an image containing the infrared 
signal for position detection being sent and, through the 

20 A/D converter 10 and switch 11, stores one frame in 
frame buffer 13. 

[0020] Lastly, in Step S7, the image address com- 
puter 16 reads out digital image signals from the frame 
buffers 12 and 13 via the read-out circuit 14, and after 

25 the comparator 15 compares the signals and finds the 
difference, finds the position of the point of origin by 
means of image processing. 
[0021 ] To wit, one frame of the image input from the 
CCD camera 5 is held in the frame buffer 12 in advance, 

30 and once the infrared signal for position detection is 
detected, the position of the portable game terminal 3 is 
detected by taking the difference between the CCD 
camera input image prior to the infrared signal being 
sent, which was stored in frame buffer 12, and the CCD 

35 camera input image after the infrared signal is sent, 
which was stored in frame buffer 13. 
[0022] In addition, the infrared signal also contains 
the identification number of the portable game terminal 
3, so it is possible to determine which portable game 

40 terminal is at which position. 

[0023] The image address computer 16 estimates 
the actual spatial coordinates from the CCD camera 
image coordinates by means of an algorithm that can be 
explained using Fig. 4 below. Here, the center of the 

45 screen of the monitor device (TV) 6 is the origin of a TV 
coordinate system having the Tz axis in the direction 
perpendicular to the screen of the TV 5. a Tx axis to the 
right when facing the screen and a Ty axis in the upward 
direction. 

so [0024] Assume that the screen direction of TV 6 
and the line-of -sight direction of CCD camera 5 are par- 
allel and that the CCD camera 5 is positioned exactly at 
the origin of the TV coordinate system. Taking the 
center of the lens of CCD camera 5 to be the origin, the 

55 Cz axis is in the forward direction perpendicular to the 
screen, the Cx axis is the~dlrection to trie right when 
looking at the CCD camera 5 and the Cy axis is in the 
upward direction. In addition, d is the distance from the 
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origin of the CCD camera coordinate system to the ori- 
gin of the TV coordinate system. 
[0025] In the case of one CCD camera, it is only 
possible to determine the two-dimensional position of 
the portable game terminal 3, so in order to limit the s 
degrees of freedom, the portable game terminal 3 is 
assumed to be present in a plane separated by a dis- 
tance a from the TV. 

[0026] Typically, if distortion of the lens is ignored, 
positions on a plane perpendicular to the z axis of the to 
camera 5 are linearly mapped to the CCD. Thereby, if 
the portable game terminal moves within the aforemen- 
tioned plane, there is a linear mapping between the 
coordinate system of the TV 6 and the coordinate sys- 
tem of the CCD camera 5. In the case of Fig. 4, it is suf- 75 
ficient to consider only the planar movement of d with 
respect to the y coordinate. 

[0027] For this reason, the image address compu- 
ter 16 can calculate Tz=Cz, Ty=Cy+d and Tx=Cx. 
Therefore, in the video game player machine 1 the posi- 20 
tion of the player holding the portable game terminal 3 
can be calculated. Then, it is possible to display the 
images generated by the execution of the program 
along a line of sight that assumes that position to be the 
center, so a point of sight optimized to the player can be 25 
given. 

[0028] tt should be noted that the input position 
detection device and entertainment system according 
to the present invention is in no way limited to the afore- 
mentioned application to a video game player device or 30 
machine 1. As shown in Fig. 5, it could also be applied 
to a video game player machine 20 wherein the identifi- 
cation number of a portable game terminal 3 that the 
game machine main unit 2 wishes to know is sent with 
an infrared signal. Arrow A indicates an infrared query 35 
signal; arrow B designates an infrared response signal. 
[0029] The video game player machine or device 20 
operates in accordance with the flowchart shown in Fig. 
6. To wit, in Step S1 1 , the game machine main unit 21 
uses an infrared signal to send the identification number 40 
of the portable game terminal 3 whose position it wishes 
to know, together with a send instruction. In Step S12, 
the portable game terminal 3 uses its own CPU to 
decode the signal received in the aforementioned infra- 
red signal, and determine if the identification number in 45 
the query is equal to its own identification number. 
[0030] In this Step S12, if the unit's own identifica- 
tion number is contained in the infrared query signal, 
then in Step S1 3 the portable game terminal 3 sends an 
infrared signal containing its own identification number sc 
as a response signal. 

[0031] Upon doing this, in Step S14, the infrared 
input/output device 4 connected to the game machine 
main unit 2 detects the aforementioned infrared signal. 
[0032] Then, from the identification number con- 55 
tained in the infrared'signal detected in Step S15. the 
signal from the portable game terminal 3 is determined, 
and the image picked up from the CCD camera 5 is con- 



verted to digital image signals by the A/D converter 10 
and then one frame is stored in the frame buffer 12. 
[0033] In Step S16, when the infrared signal for 
position detection from the portable game terminal 3 is 
sent for a length of time equal to or longer than one 
frame, in Step S17 the game machine main unit 2 uses 
the CCD camera 5 to pick up an image containing the 
infrared signal for position detection being sent and, 
through the A/D converter 10 and switch 1 1 , stores one 
frame in frame buffer 13. 

[0034] Lastly, in Step S1 8, the image address com- 
puter 16 reads out digital image signals from the frame 
buffers 12 and 13 via the read-out circuit 14, and after 
the comparator 15 compares the signals and finds the 
difference, finds the position of the point of origin by 
means of image processing. 
[0035] To wit, one frame of the image input from the 
CCD camera 5 is held in the frame buffer 12 in advance, 
and once the infrared signal for position detection is 
detected, the position of the portable game terminal 3 is 
detected by taking the difference between the CCD 
camera 5 input image prior to the infrared signal being 
sent, which was stored in frame buffer 12, and the CCD 
camera input image after the infrared signal is sent, 
which was stored in frame buffer 13. 
[0036] Here, if there is one light emitting diode 8 
attached to the portable game terminal 3, then two- 
dimensional position detection is possible. In addition, if 
there are two light emitting diodes attached at separate 
locations, it is possible to estimate the distance from the 
CCD camera 5 or detect the angle of rotation with 
respect to the screen. In addition, if there are three or 
more light-emitting diodes at separate locations, then it 
is possible to estimate the three-dimensional position 
and orientation of the portable game terminal. If neces- 
sary, the light timing of the individual light-emitting 
diodes can be shifted so that they are contained in sep- 
arate frames so that it is possible to identify the light- 
emitting diode. 

[0037] Moreover, the aforementioned video game 
players device or machine 1 and 20 can detect the posi- 
tion of the game player using the results of detecting the 
input position of the portable game terminal 3 calculated 
by the image address computer 16, and display a 
screen generated by the execution of the program along 
a line of site using that position as the center. 
[0038] In a normal game, it is assumed that the 
player is viewing the screen of the TV 6 from directly in 
front of the screen. By converting to a viewpoint that 
puts the position of the player at the center, it is possible 
to set the viewpoint appropriately when playing on a 
large screen. 

[0039] In addition, as shown in Fig. 7, in the case of 
head-to-head play between operators A and B (21 and 
22), it is possible to set appropriate viewpoints. In Fig. 7, 
the screen 23 A for operator A 21 and the screen 23 B 
for operator B 22 on the screen of the TV 6 are each 
given viewpoints appropriate to each operator. 
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[0040] In addition, by using the input positions 
detected by means of the present invention, the direc- 
tion of the gaze of characters on the screen can be 
pointed toward the position of the portable game termi- 
nal, so life-like eye contact can be expressed. In addi- 5 
tion, it is possible to provide input of movement by 
compensating for the movement of the portable game 
terminal, so it can be used instead of a keypad or joy- 
stick or the like. 

[0041] In this manner, by means of a system con- 10 
sisting of the video game players device or machine 1 
and 20 in the aforementioned embodiment, along with a 
plurality of portable game terminals, it is possible to 
determine the positions of the operators from the posi- 
tions of the portable game terminals serving as input 15 
devices, so various types of feedback can be achieved. 
[0042] In addition, by inserting an identification 
number into the infrared signal, it is possible to deter- 
mine the position and also which input device is there. 
[0043] Moreover, by taking the difference of 20 
images, it is possible to measure the position accurately 
with a simple process. If necessary, a filter through 
which infrared light passes may be attached to the CCD 
camera to permit measurements to be made at an even 
higher S/N ratio. 25 
[0044] By means of the present invention, it is pos- 
sible to provide an input position detection device that 
can accurately and easily detect the input position of an 
input device, and also an entertainment system that can 
detect the position of a subunit and display an image so 
generated by the execution of a program from a view- 
point with that position as its center. 

Claims 

35 

1. An input position detection device for detecting an 
input position by means of an input device having 
first infrared transceiving means, comprising: 

second infrared transceiving means for send- 40 
ing infrared signals to and receiving infrared 
signals from said first infrared transceiving 
means of said input device, and 
image pickup means for picking up an image in 
a stipulated direction, wherein 45 
said image pickup means and said second 
infrared transceiving means are disposed adja- 
cently each other and point in a stipulated 
direction, and an input position of said input 
device in said stipulated direction or a vicinity so 
thereof is detected based on a difference 
between an image picked up by said image 
pickup means that contains an infrared signal 
for position detection from said second infrared 
transceiving means of said input device in said 55 
stipulated direction or the vicinity thereof and 
an immediately previous image which does not 
contain an infrared signal for position detection. 



2. An entertainment system comprising: 

a main unit which has program execution func- 
tions; 

a subunit including an interface and first infra- 
red transceiving means for connecting to said 
main unit; 

second infrared transceiving means for send- 
ing infrared signals to and receiving infrared 
signals from said first infrared transceiving 
means of said subunit, and 
image pickup means for picking up an image in 
a stipulated direction, disposed adjacently to 
said second infrared transceiving means and 
pointing in a stipulated direction, 
wherein an input position of said subunit in said 
stipulated direction or vicinity thereof is 
detected based on a difference between an 
image picked up by said image pickup means 
that contains an infrared signal for position 
detection from said first infrared transceiving 
means of said subunit in said stipulated direc- 
tion or vicinity thereof and an immediately pre- 
vious image that does not contain an infrared 
signal for position detection. 

3. The entertainment system recited in Claim 2, and 
further comprising display means, wherein said 
main unit is connected to said display means for 
displaying images generated by the execution of 
programs, and said second infrared transceiving 
means and said image pickup means are disposed 
near said display means. 

4. The entertainment system recited in Claim 2, 
wherein said subunit has a unique identification 
number and information about said identification 
number is contained in infrared light emitted 
towards said main unit. 

5. The entertainment system recited in Claim 2, 
wherein said subunit has a unique identification 
number and when the identification number is sent 
from said main unit to said subunit by means of 
infrared signals, a determination is made as to 
whether or not said identification number is equal to 
a unique identification number. 

6. The entertainment system recited in Claim 2, 
wherein in that said second infrared transceiving 
means comprises a plurality of units. 

7. The entertainment system recited in Claim 6, 
wherein said plurality of infrared transceiving 
means include light-emitting diodes and are set 
such that light emissionliming of each light-emitting 
diode is offset. 
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8. The entertainment system recited in Claim 2, 
wherein in the reception of an infrared signal by 
said first* infrared transceiving means of said subu- 
nit, an image that does not contain an infrared sig- 
nal for detection of said immediately previous 5 
position is obtained and at the same time, an image 
that contains an infrared signal for position detec- 
tion is obtained in the receipt of an infrared signal at 
fixed times by said first infrared transceiving means 

of said subunit. 1C 

9. An entertainment system comprising: 

a CCD camera; 

an infrared input/output device; 15 
two frame buffers; 

an A/D converter for converting an image 
picked up by said CCD camera into a digital 
signal; 

a switch which when an infrared signal for posi- 20 
tion detection is supplied from said infrared 
input/output device, provides the output of said 
A/D converter to one of said two frame buffers; 
a comparator for comprising contents of said 
two frame buffers; and 25 
an image address computer for estimating 
actual spatial coordinates from image coordi- 
nates of said CCD camera. 

1 0. The entertainment system recited in Claim 9 ( fur- 30 
ther comprising a monitor device wherein the esti- 
mation of the actual spatial coordinates from the 
image coordinates of said CCD camera is per- 
formed by a conversion of said image coordinates 

to coordinates of said monitor device. 35 
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BUTTON OF PORTABLE GAME 
TERMINAL IS PRESSED. 



PORTABLE GAME TERMINAL SENDS 
AN INFRARED SIGNAL CONTAINING 
ITS OWN IDENTIFICATION NUMBER. 



INFRARED INPUT/OUTPUT 
DEVICE DETECTS THE 
INFRARED SIGNAL 



SIGNAL FROM THE PORTABLE GAME 
TERMINAL IS DETERMINED FROM THE 
IDENTIFICATION NUMBER AND ONE 
FRAME OF THE IMAGE IS STORED. 



PORTABLE GAME TERMINAL SENDS 
INFRARED SIGNAL FOR POSITION 

DETECTIONjTURNED ON AFTER THE 
TIME FOR 1 FRAME OR MORE). 



ONE FRAME OF THE IMAGE WITH 
THE INFRARED SIGNAL PRESENT 
IS STORED. 



THE DIFFERENCE BETWEEN THE TWO 
IMAGES IS OBTAINED TO FIND THE 
POSITION OF THE SENDING POINT 
BY IMAGE PROCESSING. 
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GAME MACHINE MAIN UNIT USES AN INFRARED 
SIGNAL TO SEND THE IDENTIFICATION NUMBER 
OF THE PORTABLE GAME TERMINAL WHOSE 
POSITION IT WISHES TO KNOW, TOGETHER 
WITH A SEND INSTRl 
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SIGNAL FROM THE PORTABLE GAME 
TERMINAL IS DETERMINED FROM THE 
IDENTIFICATION NUMBER AND ONE 
FRAME OF THE IMAGE IS STORED. 



PORTABLE GAME TERMINAL SENDS 
INFRARED SIGNAL FOR POSITION 
DETECnONfTURNED ON AFTER THE 
TIME FORI FRAME OR MORE). 



ONE FRAME OF THE IMAGE WITH 
THE INFRARED SIGNAL PRESENT 
ISSTOREO 



THE DIFFERENCE BETWEEN THE TWO 
IMAGES IS OBTAINED TO RND THE 
POSITION OF THE SENDING POINT 
BY IMAGE PROCESSING. 
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